In 2011, Judea Pearl received the Turing Award, considered the Nobel Prize in Computing, for fundamental contributions to artificial intelligence through the development of a calculus for probabilistic and causal reasoning. It includes pioneering the development of causal discovery algorithms. These computer algorithms can analyze large multivariate datasets and automatically discover the causal relationships among the constituent variables. They have been widely used in many critical fields such as medicine and economics to support decisions. However, to our knowledge, they have not been leveraged in law. This paper attempts to alleviate this gap by investigating whether causal discovery algorithms can be leveraged for automated generation of legal arguments. To that end, a novel legal dataset is prepared by identifying 17 legal concepts, such as physical assault and property dispute. A curated collection of 150 homicide cases are annotated with these concepts, e.g., a case is annotated with physical assault only if a physical assault had been reported in that case. Subsequently, a selected set of widely-used causal discovery algorithms is applied to the annotated dataset to discover the causal relationships between the legal concepts. Additionally, the degrees of belief associated with the discovered relationships are quantified in mathematical probabilities. It is shown that some of the causal relationships help generate viable legal arguments, e.g., if one could establish that a physical assault has not taken place during a homicide, it should be a sufficient condition (with probability 1) to establish that the homicide has not been committed due to a property-related dispute. Thus, this paper shows that causal discovery algorithms can be helpful in generating legal arguments, opening up avenues for promising future endeavors.

Large Language Models (LLMs) demonstrate potential in complex legal tasks like argument generation, yet their reliability remains a concern. Building upon pilot work assessing LLM generation of 3-ply legal arguments using human evaluation, this paper introduces an automated pipeline to evaluate LLM performance on this task, specifically focusing on faithfulness (absence of hallucination), factor utilization, and appropriate abstention. We define hallucination as the generation of factors not present in the input case materials and abstention as the model's ability to refrain from generating arguments when instructed and no factual basis exists. Our automated method employs an external LLM to extract factors from generated arguments and compares them against the ground-truth factors provided in the input case triples (current case and two precedent cases). We evaluated eight distinct LLMs on three tests of increasing difficulty: 1) generating a standard 3-ply argument, 2) generating an argument with swapped precedent roles, and 3) recognizing the impossibility of argument generation due to lack of shared factors and abstaining. Our findings indicate that while current LLMs achieve high accuracy (over 90%) in avoiding hallucination on viable argument generation tests (Tests 1 & 2), they often fail to utilize the full set of relevant factors present in the cases. Critically, on the abstention test (Test 3), most models failed to follow instructions to stop, instead generating spurious arguments despite the lack of common factors. This automated pipeline provides a scalable method for assessing these crucial LLM behaviors, highlighting the need for improvements in factor utilization and robust abstention capabilities before reliable deployment in legal settings. Link: https://lizhang-aiandlaw.github.io/An-Automated-Pipeline-for-Evaluating-LLM-Generated-3-ply-Case-Based-Legal-Arguments/

In this paper, we conduct an empirical analysis of how large language models (LLMs), specifically GPT-4, interpret constitutional principles in complex decision-making scenarios. We examine rulings from the Italian Constitutional Court on bioethics issues that involve trade-offs between competing values and compare model-generated legal arguments on these issues to those presented by the State, the Court, and the applicants. Our results indicate that GPT-4 consistently aligns more closely with progressive interpretations of the Constitution, often overlooking competing values and mirroring the applicants' views rather than the more conservative perspectives of the State or the Court's moderate positions. Our experiments reveal a distinct tendency of GPT-4 to favor progressive legal interpretations, underscoring the influence of underlying data biases. We thus underscore the importance of testing alignment in real-world scenarios and considering the implications of deploying LLMs in decision-making processes.

Identifying, classifying, and analyzing arguments in legal discourse has been a prominent area of research since the inception of the argument mining field. However, there has been a major discrepancy between the way natural language processing (NLP) researchers model and annotate arguments in court decisions and the way legal experts understand and analyze legal argumentation. While computational approaches typically simplify arguments into generic premises and claims, arguments in legal research usually exhibit a rich typology that is important for gaining insights into the particular case and applications of law in general. We address this problem and make several substantial contributions to move the field forward. First, we design a new annotation scheme for legal arguments in proceedings of the European Court of Human Rights (ECHR) that is deeply rooted in the theory and practice of legal argumentation research. Second, we compile and annotate a large corpus of 373 court decisions (2.3M tokens and 15k annotated argument spans). Finally, we train an argument mining model that outperforms state-of-the-art models in the legal NLP domain and provide a thorough expert-based evaluation. All datasets and source codes are available under open lincenses at https://github.com/trusthlt/mining-legal-arguments.

Artificial Intelligence (AI) is gradually and inexorably entering into the legal profession. There is the use of Natural Language Processing (NLP), which we already experience in everyday ordinary interaction with Alexa and Siri and has been increasingly added into various LegalTech systems such as used for contract management, e-Discovery, and the like. Another avenue of AI consists of Machine Learning and Deep Learning. These computational pattern matching techniques are being used to predict court rulings and are also employed to ferret out prior relevant cases amongst a large-scale corpus of online court records. One of the most fascinating and likely law-disruptive AI technologies involves AI-based legal reasoning systems. The notion is that the AI simulates the legal argumentation precepts of human attorneys and essentially carries out a limited form of legal reasoning. Initially, these AI-based legal reasoners would be used as an aid for lawyers and jurists seeking to craft legal arguments. In this semi-autonomous mode, the AI works hand-in-hand with the human legal expert and they jointly establish a robust legal argument or legal posture. Some assert that this capability by the AI will inevitably be further advanced and we will have available fully autonomous AI-based legal reasoning systems that can act in lieu of needing any human legal guidance.

Apple doesn't allow apps that contain third-party games inside its App Store, including Microsoft's Project xCloud streaming service, Facebook Gaming (which is allowed on iOS but doesn't contain games) and GameClub. GameClub was brought up by Apple's Karen Dunn on Wednesday as an example of a competitor to Apple Arcade, the tech giant's gaming subscription service, that is allowed on iOS. But that invocation drew ire from an unexpected source: GameClub itself. The company's vice president of business development, Eli Hodapp, told The Washington Post that GameClub was rejected "well over a hundred times in our quest to getting on the App Store" and "has experienced unnecessary friction due to Apple's requirements."

The effective use of argument is, of course, central to the practice of Law, and it is important that students of Law learn this skill. We describe here the architecture of a computer-based system to enable students to practice argumentation in a regulated environment. The system makes use of the concept of a dialogue game as a means of providing the necessary rule-governed structure for the conduct of an argument between two students, or a student and a teacher. The architecture described is generic in that it can be instantiated with different forms of dialogue game. This instantiation is achieved by the use of performatives to specify the rules of the game and the semantics of operations within the Dialogue Abstract Machine that is used to implement it.

An artificial intelligence method developed by University College London computer scientists and associates has predicted the judicial decisions of the European Court of Human Rights (ECtHR) with 79% accuracy, according to a paper published Monday, Oct. 24 in PeerJ Computer Science. The method is the first to predict the outcomes of a major international court by automatically analyzing case text using a machine-learning algorithm.* "We don't see AI replacing judges or lawyers," said Nikolaos Aletras, who led the study at UCL Computer Science, "but we think they'd find it useful for rapidly identifying patterns in cases that lead to certain outcomes. It could also be a valuable tool for highlighting which cases are most likely to be violations of the European Convention on Human Rights." In developing the method, the team found that judgments by the ECtHR are highly correlated to non-legal (real-world) facts, rather than direct legal arguments, suggesting that judges of the Court are, in the jargon of legal theory, "realists" rather than "formalists."

Technology around us will provide an "augmented intelligence" that will help humans to make smarter decisions, improve business models and solve problems that were previously intractable. "The ways in which we are able to interact with computers is going to make people a lot more efficient and more effective, and build digital models." This, says Richard Paris, senior data scientist at KPMG New Zealand, is the future of digital. We are increasingly seeing the digital world interact in our everyday lives, says Paris, who spoke at the inaugural KPMG Technology Series in Auckland. People interact with smartphones and these devices are becoming our intelligent assistants.

Technology around us will provide an "augmented intelligence" that will help humans to make smarter decisions, improve business models and solve problems that were previously intractable. "The ways in which we are able to interact with computers is going to make people a lot more efficient and more effective, and build digital models." This, says Richard Paris, senior data scientist at KPMG New Zealand, is the future of digital. We are increasingly seeing the digital world interact in our everyday lives, says Paris, who spoke at the inaugural KPMG Technology Series in Auckland. People interact with smartphones and these devices are becoming our intelligent assistants.